The present invention relates to a window display device for displaying windows on the screen of an information processing apparatus, and in particular to a device for displaying windows in a virtual three-dimensional (3D) space.
Multiple window display devices are used to increase the performance of the man-machine interface in computer information terminals and the like.
One example of a prior art technique is disclosed in Japanese Laid-Open Patent No 6-222899. Here, a multiple window display device improves display efficiency by displaying as many windows as possible in the limited screen space available.
FIG. 69 shows a display screen for this prior art example.
In the prior art, windows are placed in a virtual 3D space, with windows positioned deep within the 3D space reduced in size. As a result, more windows can be displayed, making the screen appear larger than it actually is.
However, in this prior art example, a window placed deep within the 3D space is displayed reduced in size, making it impossible to distinguish any of the display content of the window.
The present invention, a device for displaying windows placed in a virtual 3D space, is designed to overcome the stated problem. The object of the invention is to provide a window display device that increases the efficiency with which windows can be displayed, making the screen appear larger than it actually is, and moreover, preventing a situation in which the entire display content of a window is unreadable.
The present invention aiming to achieve this object is a window display device displaying a plurality of windows on a screen. The window display device includes the following. A display data holding unit holds display data representing display content of the plurality of windows. A position determining unit determines a position for each window in a virtual three-dimensional (3D) space, the positions being such that the windows are not parallel to the screen. A perspective projection unit transforms the display data to project the windows onto the screen in perspective based on the determined positions. A display unit then displays the transformed display data on the screen.
This enables at least one window to be displayed inclined in the depth direction of a virtual 3D space. Even if the window is positioned deep within the 3D space, the problem apparent in the prior art, where reducing the size of the whole window by the same amount meant that all of the window display content became unreadable, can be avoided. The screen display area can also be used more effectively, and the screen made to appear larger than it actually is. Furthermore, information in the inclined window that is near to the front of the 3D space remains easily recognizable, and less display area needs to be used for the part of the window deep in the 3D space. Thus a window display apparatus capable of using the screen display area effectively, while ensuring that at least one part of the window display content is readable can be realized.
Here, the window display device further includes an input unit acquiring indication from a user, enabling the position determining unit to determine positions according to indications acquired from the input unit. This enables the user to determine the size of the entire window and the degree at which it slants in the depth direction.
The position determining means also determines 3D coordinates for each vertex of a window as the position for the window. If the window is a rectangular plane, this enables the position of the window in the 3D space to be determined simply by the coordinates for its four vertices.
The window display device further includes a window analyzing unit detecting, as a key part for each window, a part of the display content fulfilling certain conditions. Detection is performed based on display data held by the display data holding unit. This enables the position determining unit to determine the position of windows so that the key parts detected by the window analyzing unit are displayed near the front of the virtual 3D space. The window can thus be displayed inclined in the depth direction, so that the key part of the display content is near the front of the 3D space. Therefore, even if a plurality of windows are being displayed, the user can grasp the type and display content of each window at a glance.
The window analyzing unit also determines whether a title is included in the display data for each window and, if a title is included, detects a part of a window in which the title is displayed as the key part. This enables the user to easily recognize window types.
The window analyzing unit also determines whether a text and an indication for displaying the text horizontally are included in the display data for each window, and if the text and the indication are included, detects a left side of a window when viewed from the front as the key part. This enables the user to easily recognize the type and general content of a horizontally-written document displayed in a window.
The window analyzing unit also determines whether a text, and an indication for displaying the text vertically are included in the display data for each window, and if the text and the indication are included, detects a right side of a window when viewed from the front as the key part. This enables the user to easily recognize the type and general content of a vertically-written document displayed in a window.
The position determining unit includes an automatic alignment unit for determining the position of a plurality of windows so that the windows are placed in a predetermined arrangement, where at least one window is not parallel with the screen. This enables a plurality of windows that were displayed scattered around the screen to be displayed in alignment, using the screen display area more effectively and allowing all of the windows currently displayed to be viewed at a glance.
The position determining means further includes a unit determining a position according to an indication acquired from the input unit, so that a window displayed in the alignment position is newly displayed facing the front; and a unit determining a position according to an indication acquired from the input unit, so that a window displayed facing the front is returned to the alignment position. This enables a operation, in which a window faces the front when it is being operated on, and returns to the alignment position once it is no longer being operated on, to be easily performed.
The window display device further includes a window information image position changing unit editing display data stored in the display data holding unit so that a window information image is displayed near the front of the virtual 3D space. The window information image is formed from a title bar and a menu bar, and editing is performed based on a position determined by the position determining unit. This enables the title bar and menu bar for a window to be displayed near the front of the 3D space, even if the window is inclined in the depth direction. As a result, recognizing window types and performing window operations becomes easier.
The window display device further includes a window information image orientation changing unit controlling the position determining unit so that, in a window where a window information image is displayed near the front of the virtual 3D space, the window information image faces the front, while the remaining part of the window :is not parallel with the screen. This enables the title bar and menu bar to be displayed facing the front near the front of the virtual 3D space, even if the window is inclined in the depth direction. As a result, recognizing window types and performing window operations becomes easier.
The window display device further includes the following. An unreadable area determining unit determines whether an unreadable area displaying characters smaller than a specified size has been generated from display data transformed by the perspective projection unit. A scroll button position determining unit determines a position for a scroll button when an unreadable area has been determined to exist, the position determined as if the unreadable area was not displayed in the window. A scroll bar attachment unit edits the display data so that a scroll bar image, including a scroll button arranged at the determined position, is attached to the window. This enables characters and the like in an unreadable area of a window displayed inclined in the depth direction to be scrolled up into the readable display area.
The window display device further includes the following. A layer separating unit separates display data held in the display data holding unit, the display data corresponding to a window with a layered construction having a base layer and at least one upper layer, and the display data for each upper layer, but not the base layer, being separated. A window layer generating unit generates display data for a window layer corresponding to each separated layer. The position determining unit determines a position for each window layer parallel with and a specified distance from the base layer. This enables window layers to be displayed floating over the base layer so that objects displayed on each layer can be grasped at a glance, and editing of an object across windows can be easily performed.
The window display device further includes the following. A keyword searching unit searches display data held in the display data holding unit for parts that match a keyword acquired from the input unit. A window layer generating unit generates display data for a new window layer in which the keyword is arranged at positions corresponding to the matching parts, and stores the generated display data in the display data holding unit. Here, the position determining unit determines a position for the window layer parallel with and a specified distance from a display object window. This enables a window layer showing the matching parts to be displayed floating over the original window. The user""s attention is focused on the window layer only, allowing the position of the keyword to be confirmed.
The window display device further includes the following. An inter-related window specifying unit specifies a group of inter-related windows from the plurality of displayed windows. A link information storage unit generates and stores link information used to line up the group of inter-related windows in a concertina arrangement with frames of neighboring windows touching. Here, the position determining unit determines positions of the group of inter-related windows based on the link information. This enables a plurality of document pages displayed in different windows to be displayed in a linked sequence, so that, for example, windows belonging to the same document can be easily distinguished.
The window display device further includes the following. A link information changing unit changes the link information according to an indication acquired from the input means, so that the group of inter-related windows is lined up with part of the group excluded. Here, the position determining unit determines positions for the group of inter-related windows, apart from the excluded part of the group, based on the changed link information. This enables the windows that the user wants to focus on from the linked sequence of windows to be displayed.
The window display device further includes an inter-window editing unit editing display data held in the display data holding unit according to an indication acquired from the input unit, so that editing of display content can be performed between two windows in the group of inter-related windows. This enables the windows that the user wants to focus on from the linked sequence to be displayed adjacent to each other, and editing of an object performed between the two windows, so that the efficiency of operations performed spanning a plurality of windows is improved.
The display data holding unit includes a character data storage unit storing character data formed of character information displayed in the windows, and a window data storage unit storing window data formed of information concerning non-character information displayed in the windows. Here, the position placement determining unit determines a position for each window and the corresponding characters according to an indication acquired from the input unit, and the perspective projection unit includes the following. A perspective transform unit generates two-dimensional data showing the vertices and window image data by performing a perspective projection transform on the window data, the perspective projection transform being performed based on window positions determined by the position determining unit. A character data transform unit generates character image data by transforming the character data, based on character positions determined by the position determining unit and two-dimensional data showing vertices obtained by the transform performed by the perspective transform unit. A combining unit combines image data generated by the perspective transform unit and image data generated by the character data transform unit. Here, the display unit displaying image data combined by the combining unit.
The character data storage unit includes a character information storage unit storing information relating to characters displayed in the window, the information including at least character position information showing positions of the characters in virtual space, character identifying information specifying which characters should be displayed, and character size information, and a character image storage unit storing character images. Here, the window data storage unit stores graphics data, comprising window display content excluding characters, and window coordinate information showing a position of the window in the virtual 3D space, the position determining unit changes the window data and the character position information in response to an indication acquired from the input unit and the perspective transform unit performs a perspective projection transform on window data that has been changed by the position determining unit. The character data transform unit includes the following. A character position information conversion unit converts character position information changed by the position determining unit into screen position information. A character image selecting unit selects a character image from the character images stored in the character image storage unit, the character image corresponding to the character identifying information. A character image transform unit transforms the character image selected by the character image selecting unit in accordance with the result of the position changing process performed by the position determining unit. Here, the combining unit combines a character image transformed by the character image transform unit with image data generated by the perspective transform unit, so that the character image is placed at a position in the window shown by the screen position information produced by the character position information conversion unit, the window having been transformed onto the screen by the perspective transform unit.
The character image transform unit includes a shape transform unit transforming the shape of the character image using a similarity transform related to the shape of the window transformed by the perspective transform unit; and a scaling unit finding a scale based on character position information stored in the character information storage unit, and changing the size of the character image according to the scale. Here, character images are separated from the window as 2D data, their shape transformed by a similarity transform with the perspective-transformed window, and their size converted according to a scale calculated from coordinate values. After being transformed into a shape similar to the perspective transform result, the character images are combined with the perspective-transformed window. This processing occurs rather than having character images rotated and perspective-transformed after they have been developed into virtual 3D data. When compared to the prior art, in which a window including character images was rotated and then perspective-projected, the amount of processing is reduced, and performance improved. The amount of data used for the character images is also decreased, reducing the amount of memory required.
The character image transform unit calculates a scale based on character position information stored in the character information storage unit, and changes the size of the character image according to the scale. This enables only character image size to be changed by the scale found from the character position coordinates, further reducing the amount of processing required and creating a sense of depth when characters are displayed in a window as a character string.
The character image transform unit includes an aspect ratio conversion unit finding an aspect ratio based on character position information stored in the character information storage unit and changing the length to width ratio of the character image according to the aspect ratio, and a scaling unit finding a scale based on character position information stored in the character information storage unit, and changing the size of the character image according to the scale. This enables character images to be transformed using an aspect ratio, rather than simply by using a scale. As a result, a more natural sense of depth can be achieved when character images are displayed as a character string in a window than was the case when only the size of character images was changed according to a scale.
The character image storage unit stores character images on which a similarity transform has been performed for each of a plurality of representative positions for a window in the virtual 3D space, the similarity transform being related to the shape of the window produced at each of the plurality of positions as a result of a perspective projection transform. The character image selecting unit selects a character image from the character image storage unit, the character image coinciding with character identifying information stored in the character information storage unit, and with window coordinate information stored in the window data storage unit. The character image transform unit finds a scale based on character position information stored in the character information storage unit, and changes the size of the character image selected by the character image-selecting unit,.the size of the character image being changed according to the scale. This enables character images having a natural sense of depth and similar to character images that have been perspective-transformed, to be displayed simply by changing the size of pre-prepared character images.
The window display device further includes a character image readability judging unit. This unit calculates a size for a transformed character image transformed by the character image transform unit from size information stored in the character information storage unit, and a scale calculated from character position information stored in the character information storage unit. It then judges whether the transformed character image is readable by comparing the calculated size with a threshold value, and informs the character image selecting unit of the judgement result. Here, the character image storage unit further stores symbol images, the character image selecting unit selects a specified symbol image from the character image storage unit to be displayed instead of a character image if the character image readability judging unit judges that the character image is unreadable, the character image transform unit does not perform transform processing on the symbol image, and the combining unit combines the symbol image with image data generated by the perspective transform unit so that the symbol image is placed at a position in a window shown by screen position information produced by the character position information conversion unit, the window having been transformed onto the screen by the perspective transform unit. This enables a symbol to be displayed instead of characters considered to be difficult or impossible to read if transformed. As a result, processing for reading and transforming character images is shortened, reducing both processing volume and the amount of time required for processing.
The character information storage unit further stores font type information showing character font types. The character image storage unit stores character images for each font type. The window display device further includes a character image readability judging unit. This unit calculates a size for a transformed character image transformed by the character image transform unit from size information stored in the character information storage unit, and a scale calculated from character position information stored in the character information storage unit. It judges if the font type of the transformed character image is readable by comparing the calculated size with a threshold value, and informs the character image selecting unit of the judgement result. Here, when the font type of a character is judged unreadable by the character image readability judging unit, the character image selecting unit selects a character image with a specified font type, ignoring font type information stored in the character information storage unit. This enables individual fonts to be ignored in the case of character images whose display font will not be discernable if transformed. Instead these characters are displayed in an identical specified font. If cache memory is used, the cache hit rate when the font is read increases, improving processing efficiency.
The window data storage unit further stores coordinate system information showing a virtual space window coordinate system, and the window display device further includes a coordinate system converting unit for changing a scale of a coordinate axis showing screen positions, the scale being changed according to window coordinate information stored in the window data storage unit. Here, the perspective projection unit performs a perspective projection transform, converting the coordinate system to a coordinate system having a coordinate axis with the changed scale. This enables the scale of the coordinate system to be changed, so that even characters far from the rotation axis, which would be cramped and difficult to read in a conventional coordinate system, can be displayed so that they are easy to read.